{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Running This Guide:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This guide is presented as a series of Jupyter notebooks covering both Tensorflow and PyTorch using a Python runtime.\n",
    "\n",
    "If you would like to run this code yourself, you can do so using the following steps:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Setup__:\n",
    "\n",
    "The basic prerequisites you will need to use TensorRT are:\n",
    "\n",
    "- a [supported NVIDIA GPU](https://docs.nvidia.com/deeplearning/tensorrt/support-matrix/index.html#hardware-precision-matrix) (preferred compute capability 7.0 or higher - which includes INT8 precision support) \n",
    "- latest [NVIDIA GPU drivers](https://docs.nvidia.com/datacenter/tesla/tesla-installation-notes/index.html)\n",
    "- a [supported CUDA and cuDNN](https://docs.nvidia.com/deeplearning/tensorrt/support-matrix/index.html#platform-matrix) installation\n",
    "\n",
    "You can make sure your GPU environment is properly configured and check which GPU and CUDA version you are using with nvidia-smi:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!nvidia-smi "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you have Python versions 3.5 to 3.8 and CUDA 11.1 (or are in Google Colab), you can do the following to install TensorRT:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install --upgrade --index-url https://pypi.ngc.nvidia.com nvidia-tensorrt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For some of the examples you will also need pycuda, skimage, and onnx:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install pycuda onnx scikit-image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Tensorflow__:\n",
    "\n",
    "We will be using Tensorflow 2 to walk through the basic steps of deploying a TensorRT model. \n",
    "\n",
    "You can find Tensorflow installation instructions [here](), or you can use one of NVIDIA's NGC containers [here](https://ngc.nvidia.com/catalog/containers/nvidia:tensorflow). \n",
    "\n",
    "The Tensorflow examples also need keras2onnx and matplotlib:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install matplotlib keras2onnx tf2onnx"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__PyTorch__:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also use other frameworks, such as PyTorch, with TensorRT by [exporting the model in ONNX format](https://pytorch.org/docs/stable/onnx.html). For this guide, we will demonstrate this process using PyTorch.\n",
    "\n",
    "You can find PyTorch installation instructions [here](https://pytorch.org/get-started/locally/), or use one of NVIDIA's NGC containers [here](https://ngc.nvidia.com/catalog/containers/nvidia:pytorch). \n",
    "\n",
    "You will also need torchvision:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install torchvision"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Using Colab:__\n",
    "\n",
    "You can also test these notebooks out using [Google Colab](https://colab.research.google.com/), which includes both Tensorflow and PyTorch, as well as supported NVIDIA drivers. __Make sure to select a GPU hardware accelerator__ in the runtime options. Just note that TensorRT performance is best on newer gpus, Colab often has trouble with reduced precision inference, and you will have to use an older version of TensorRT.\n",
    "\n",
    "At the time of writing, Colab is on an old version of CUDA (10.1) that TensorRT 7 does not support. You will need to install TensorRT 6 - at the time of writing the following works to install it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "version=\"6.0.1-1+cuda10.1\"\n",
    "wget https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/nvidia-machine-learning-repo-ubuntu1804_1.0.0-1_amd64.deb\n",
    "\n",
    "dpkg -i nvidia-machine-learning-repo-*.deb\n",
    "apt-get update\n",
    "\n",
    "sudo apt-get install libnvinfer6=${version} libnvonnxparsers6=${version} libnvparsers6=${version} libnvinfer-plugin6=${version} libnvinfer-dev=${version} libnvonnxparsers-dev=${version} libnvparsers-dev=${version} libnvinfer-plugin-dev=${version} python-libnvinfer=${version} python3-libnvinfer=${version}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__TensorRT Support:__\n",
    "\n",
    "TensorRT support for NVIDIA GPUs is determined by their __compute capability__. You can check the compute cabapility of your card on the [NVIDIA website](https://developer.nvidia.com/cuda-gpus).\n",
    "\n",
    "TensorRT supports different feautures depending on your compute capability. Higher compute capabilities allow additional TensorRT optimizations, like reduced precision inference. You can check which TensorRT features are supported by your compute capability in the [TensorRT documentation](https://docs.nvidia.com/deeplearning/tensorrt/support-matrix/index.html#hardware-precision-matrix).\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Next Steps:__\n",
    "\n",
    "Now, start by opening [1. Introduction.ipynb](./1.%20Introduction.ipynb) and proceed through the notebook!\n",
    "\n",
    "The notebooks included with this guide are:\n",
    "- [1. Introduction.ipynb](./1.%20Introduction.ipynb)\n",
    "- [2. Using the TF-TRT Tensorflow Integration.ipynb](./2.%20Using%20the%20Tensorflow%20TensorRT%20Integration.ipynb)\n",
    "- [3. Using Tensorflow 2 through ONNX.ipynb](./3.%20Using%20Tensorflow%202%20through%20ONNX.ipynb)\n",
    "- [4. Using PyTorch through ONNX.ipynb](./4.%20Using%20PyTorch%20through%20ONNX.ipynb)\n",
    "- [5. Understanding TensorRT Runtimes.ipynb](./5.%20Understanding%20TensorRT%20Runtimes.ipynb)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
